AWS DataSyncタスク作成時にENIが複数作成される意味を考えてみた
DataSyncタスクのENIが複数あるのは可用性向上のためではなさそう
君はどうして複数作成されるんだ
こんにちは、のんピ(@non____97)です。
皆さんはAWS DataSyncタスク作成時にENIが複数作成される意味が気になったことはありますか? 私はあります。
DataSyncのタスクを作成すると、エージェントの有無に応じて4つもしは2つのENIが作成されます。
- エージェントあり
| Location | Network interfaces created by default | Where network interfaces are created when using a public or FIPS endpoint | Where network interfaces are created when using a private (VPC) endpoint |
|---|---|---|---|
| Amazon S3 | 4 | N/A (network interfaces aren't needed because DataSync communicates directly with the S3 bucket) | The subnet you specified when activating your DataSync agent. |
| Amazon EFS | 4 | The subnet you specify when creating the Amazon EFS location. | The subnet you specify when creating the Amazon EFS location. |
| Amazon FSx for Windows File Server | 4 | The same subnet as the preferred file server for the file system. | The same subnet as the preferred file server for the file system. |
| Amazon FSx for Lustre | 4 | The same subnet as the file system. | The same subnet as the file system. |
| Amazon FSx for OpenZFS | 4 | The same subnet as the file system. | The same subnet as the file system. |
| Amazon FSx for NetApp ONTAP | 4 | The same subnet as the file system. | The same subnet as the file system. |
- エージェントなし
| Location | Network interfaces created by default | Where network interfaces are created |
|---|---|---|
| Amazon S3 | N/A (network interfaces aren't needed because DataSync communicates directly with the S3 bucket) | |
| Amazon EFS | 2 | The subnet you specify when creating the Amazon EFS location. |
| FSx for Windows File Server | 2 | The same subnet as the preferred file server for the file system. |
| FSx for Lustre | 2 | The same subnet as the file system. |
| FSx for OpenZFS | 2 | The same subnet as the file system. |
| FSx for ONTAP | 2 | The same subnet as the file system. |
複数ENIが作成される意味を想像してパッと思いつくのは可用性の向上です。片方のENIに何らかの障害が発生しても継続してデータ転送するような動きをするのか気になります。
また、ENIがActive/Activeで動作するのかも気になります。データ転送時に全てのENIで通信が発生しているのか、それとも1つのENIを除いて他のENIは通常時は動作せず、障害発生時にフェイルオーバーするような動きをするのか気になります。
気になってお盆休みで実家に帰っている間も頭に残っていたので検証します。
いきなりまとめ
- AWS DataSyncタスクのENIはActive/Activeで動作する
- セキュリティグループで片方のENIの通信を制限するとタスクが異常終了する
- ENIが足りない場合はDataSyncタスクのステータスが起動中の時にENIが作成される
- DataSyncタスクのステータスが転送中の時にENIを削除すると以降の転送が行われなくなる
検証の環境
検証の環境は以下の通りです。
EFS上に作成したファイルをS3バケットにDataSyncで転送します。
こちらの環境は全てAWS CDKでデプロイします。
使用したコードは以下リポジトリに保存しています。
テスト用ファイルの作成
テスト用ファイルを作成します。
EC2インスタンスにアクセスして、EFSをマウントしているディレクトリにランダムなバイナリデータを複数生成します。
# EFSをマウントしていることを確認
$ df -hT
Filesystem Type Size Used Avail Use% Mounted on
devtmpfs devtmpfs 471M 0 471M 0% /dev
tmpfs tmpfs 479M 0 479M 0% /dev/shm
tmpfs tmpfs 479M 352K 478M 1% /run
tmpfs tmpfs 479M 0 479M 0% /sys/fs/cgroup
/dev/nvme0n1p1 xfs 8.0G 1.6G 6.5G 19% /
fs-0bf8451bb6073b581.efs.us-east-1.amazonaws.com:/ nfs4 8.0E 0 8.0E 0% /mnt/efs
# ランダムなバイナリファイルを30個作成
$ for i in {1..30}; do
> sudo dd if=/dev/urandom of=/mnt/efs/random_block_file_"${i}" bs=1M count=128
> done
128+0 records in
128+0 records out
134217728 bytes (134 MB) copied, 0.806809 s, 166 MB/s
128+0 records in
128+0 records out
134217728 bytes (134 MB) copied, 1.02532 s, 131 MB/s
128+0 records in
128+0 records out
.
.
(中略)
.
.
134217728 bytes (134 MB) copied, 1.19255 s, 113 MB/s
128+0 records in
128+0 records out
134217728 bytes (134 MB) copied, 1.18627 s, 113 MB/s
# ファイルが作成されたことを確認
$ ls -lrt /mnt/efs/
total 3932160
-rw-r--r-- 1 root root 134217728 Aug 15 06:55 random_block_file_1
-rw-r--r-- 1 root root 134217728 Aug 15 06:55 random_block_file_2
-rw-r--r-- 1 root root 134217728 Aug 15 06:55 random_block_file_3
-rw-r--r-- 1 root root 134217728 Aug 15 06:55 random_block_file_4
-rw-r--r-- 1 root root 134217728 Aug 15 06:55 random_block_file_5
-rw-r--r-- 1 root root 134217728 Aug 15 06:55 random_block_file_6
-rw-r--r-- 1 root root 134217728 Aug 15 06:55 random_block_file_7
-rw-r--r-- 1 root root 134217728 Aug 15 06:55 random_block_file_8
-rw-r--r-- 1 root root 134217728 Aug 15 06:55 random_block_file_9
-rw-r--r-- 1 root root 134217728 Aug 15 06:55 random_block_file_10
-rw-r--r-- 1 root root 134217728 Aug 15 06:55 random_block_file_11
-rw-r--r-- 1 root root 134217728 Aug 15 06:55 random_block_file_12
-rw-r--r-- 1 root root 134217728 Aug 15 06:55 random_block_file_13
-rw-r--r-- 1 root root 134217728 Aug 15 06:55 random_block_file_14
-rw-r--r-- 1 root root 134217728 Aug 15 06:55 random_block_file_15
-rw-r--r-- 1 root root 134217728 Aug 15 06:55 random_block_file_16
-rw-r--r-- 1 root root 134217728 Aug 15 06:55 random_block_file_17
-rw-r--r-- 1 root root 134217728 Aug 15 06:55 random_block_file_18
-rw-r--r-- 1 root root 134217728 Aug 15 06:55 random_block_file_19
-rw-r--r-- 1 root root 134217728 Aug 15 06:55 random_block_file_20
-rw-r--r-- 1 root root 134217728 Aug 15 06:55 random_block_file_21
-rw-r--r-- 1 root root 134217728 Aug 15 06:55 random_block_file_22
-rw-r--r-- 1 root root 134217728 Aug 15 06:55 random_block_file_23
-rw-r--r-- 1 root root 134217728 Aug 15 06:55 random_block_file_24
-rw-r--r-- 1 root root 134217728 Aug 15 06:55 random_block_file_25
-rw-r--r-- 1 root root 134217728 Aug 15 06:55 random_block_file_26
-rw-r--r-- 1 root root 134217728 Aug 15 06:55 random_block_file_27
-rw-r--r-- 1 root root 134217728 Aug 15 06:55 random_block_file_28
-rw-r--r-- 1 root root 134217728 Aug 15 06:55 random_block_file_29
-rw-r--r-- 1 root root 134217728 Aug 15 06:55 random_block_file_30
# 作成されたファイルの合計サイズを確認
$ du -sh /mnt/efs/
3.8G /mnt/efs/
DataSyncタスクの実行 (1回目)
それではDataSyncタスクを実行します。
DataSyncタスクのENIは2つありました。
$ aws ec2 describe-network-interfaces \
--filters Name=vpc-id,Values=vpc-0bc13773f6bcc9506 \
--query "NetworkInterfaces[].[NetworkInterfaceId, Description, PrivateIpAddresses, Attachment, Groups, AvailabilityZone, SubnetId]"
[
[
"eni-0944e70a73544a760",
"EFS mount target for fs-0bf8451bb6073b581 (fsmt-0463bd620e0884896)",
[
{
"Primary": true,
"PrivateDnsName": "ip-10-0-1-55.ec2.internal",
"PrivateIpAddress": "10.0.1.55"
}
],
{
"AttachmentId": "ela-attach-026a19a4058224b84",
"DeleteOnTermination": false,
"DeviceIndex": 1,
"InstanceOwnerId": "amazon-aws",
"Status": "attached"
},
[
{
"GroupName": "EfsStack-EFSFileSystemEfsSecurityGroup5F3FED7B-1J69W17R1MWCH",
"GroupId": "sg-065656efe6cc42677"
}
],
"us-east-1b",
"subnet-0917e56a46b60966f"
],
[
"eni-0f5d19add01d21dba",
"",
[
{
"Primary": true,
"PrivateDnsName": "ip-10-0-1-5.ec2.internal",
"PrivateIpAddress": "10.0.1.5"
}
],
{
"AttachTime": "2022-08-15T05:53:16+00:00",
"AttachmentId": "eni-attach-044407d2f5056acb9",
"DeleteOnTermination": true,
"DeviceIndex": 0,
"NetworkCardIndex": 0,
"InstanceId": "i-0469068d54d7aeadb",
"InstanceOwnerId": "<AWSアカウントID>",
"Status": "attached"
},
[
{
"GroupName": "EfsStack-EC2InstanceInstanceSecurityGroup6CAFB487-K4RIQM1F0WIB",
"GroupId": "sg-0e64316320498bd21"
}
],
"us-east-1a",
"subnet-0e9dd33d8333d9601"
],
[
"eni-0a1f10f7b3e31df89",
"datasync client for loc-070033dbd5d040458 (us-east-1:task-0ada18d8a999c08a0)",
[
{
"Primary": true,
"PrivateDnsName": "ip-10-0-1-41.ec2.internal",
"PrivateIpAddress": "10.0.1.41"
}
],
null,
[
{
"GroupName": "EfsStack-EC2InstanceInstanceSecurityGroup6CAFB487-K4RIQM1F0WIB",
"GroupId": "sg-0e64316320498bd21"
}
],
"us-east-1a",
"subnet-07b506c0f2d7f9fff"
],
[
"eni-01e89bc06f8bbb7c3",
"datasync client for loc-070033dbd5d040458 (us-east-1:task-0ada18d8a999c08a0)",
[
{
"Primary": true,
"PrivateDnsName": "ip-10-0-1-39.ec2.internal",
"PrivateIpAddress": "10.0.1.39"
}
],
null,
[
{
"GroupName": "EfsStack-EC2InstanceInstanceSecurityGroup6CAFB487-K4RIQM1F0WIB",
"GroupId": "sg-0e64316320498bd21"
}
],
"us-east-1a",
"subnet-07b506c0f2d7f9fff"
],
[
"eni-07e27bf8c7fc6cf7e",
"EFS mount target for fs-0bf8451bb6073b581 (fsmt-0acc33de4179e60dc)",
[
{
"Primary": true,
"PrivateDnsName": "ip-10-0-1-38.ec2.internal",
"PrivateIpAddress": "10.0.1.38"
}
],
{
"AttachmentId": "ela-attach-078b401de82a9a60a",
"DeleteOnTermination": false,
"DeviceIndex": 1,
"InstanceOwnerId": "amazon-aws",
"Status": "attached"
},
[
{
"GroupName": "EfsStack-EFSFileSystemEfsSecurityGroup5F3FED7B-1J69W17R1MWCH",
"GroupId": "sg-065656efe6cc42677"
}
],
"us-east-1a",
"subnet-07b506c0f2d7f9fff"
]
]
まずは、普通に実行します。
少し待つと、実行ステータスが成功になりました。
転送先のS3バケットを確認すると確かにEFS内に作成したファイルが転送されていました。
DataSyncのログからも転送が正常に行われたことが確認できます。
[INFO] Request to start task-0ada18d8a999c08a0. [INFO] Execution exec-0e89cdb8072a8bf41 started. [INFO] Started logging in destination hostId: host-04a8b4d757b8780f7 for Execution exec-0e89cdb8072a8bf41 [INFO] Started logging in destination hostId: host-065ef904de256d10a for Execution exec-0e89cdb8072a8bf41 [NOTICE] Transferred file /random_block_file_7, 134217728 bytes [NOTICE] Transferred file /random_block_file_8, 134217728 bytes [NOTICE] Transferred file /random_block_file_23, 134217728 bytes [NOTICE] Transferred file /random_block_file_15, 134217728 bytes [NOTICE] Transferred file /random_block_file_14, 134217728 bytes [NOTICE] Transferred file /random_block_file_21, 134217728 bytes [NOTICE] Transferred file /random_block_file_25, 134217728 bytes [NOTICE] Transferred file /random_block_file_13, 134217728 bytes [NOTICE] Transferred file /random_block_file_12, 134217728 bytes [NOTICE] Transferred file /random_block_file_16, 134217728 bytes [NOTICE] Transferred file /random_block_file_30, 134217728 bytes [NOTICE] Transferred file /random_block_file_29, 134217728 bytes [NOTICE] Transferred file /random_block_file_11, 134217728 bytes [NOTICE] Transferred file /random_block_file_17, 134217728 bytes [NOTICE] Transferred file /random_block_file_22, 134217728 bytes [NOTICE] Transferred file /random_block_file_5, 134217728 bytes [NOTICE] Transferred file /random_block_file_3, 134217728 bytes [NOTICE] Transferred file /random_block_file_1, 134217728 bytes [NOTICE] Transferred file /random_block_file_18, 134217728 bytes [NOTICE] Transferred file /random_block_file_27, 134217728 bytes [NOTICE] Transferred file /random_block_file_28, 134217728 bytes [NOTICE] Transferred file /random_block_file_26, 134217728 bytes [NOTICE] Transferred file /random_block_file_20, 134217728 bytes [NOTICE] Transferred file /random_block_file_4, 134217728 bytes [NOTICE] Transferred file /random_block_file_24, 134217728 bytes [NOTICE] Transferred file /random_block_file_6, 134217728 bytes [NOTICE] Transferred file /random_block_file_10, 134217728 bytes [NOTICE] Transferred file /random_block_file_9, 134217728 bytes [NOTICE] Transferred file /random_block_file_19, 134217728 bytes [NOTICE] Transferred file /random_block_file_2, 134217728 bytes [NOTICE] Transferred directory metadata / [NOTICE] Verified directory / [NOTICE] Verified file /random_block_file_10, 134217728 bytes [NOTICE] Verified file /random_block_file_13, 134217728 bytes [NOTICE] Verified file /random_block_file_15, 134217728 bytes [NOTICE] Verified file /random_block_file_16, 134217728 bytes [NOTICE] Verified file /random_block_file_19, 134217728 bytes [NOTICE] Verified file /random_block_file_21, 134217728 bytes [NOTICE] Verified file /random_block_file_22, 134217728 bytes [NOTICE] Verified file /random_block_file_24, 134217728 bytes [NOTICE] Verified file /random_block_file_27, 134217728 bytes [NOTICE] Verified file /random_block_file_28, 134217728 bytes [NOTICE] Verified file /random_block_file_3, 134217728 bytes [NOTICE] Verified file /random_block_file_30, 134217728 bytes [NOTICE] Verified file /random_block_file_5, 134217728 bytes [NOTICE] Verified file /random_block_file_6, 134217728 bytes [NOTICE] Verified file /random_block_file_9, 134217728 bytes [NOTICE] Verified file /random_block_file_1, 134217728 bytes [NOTICE] Verified file /random_block_file_11, 134217728 bytes [NOTICE] Verified file /random_block_file_12, 134217728 bytes [NOTICE] Verified file /random_block_file_14, 134217728 bytes [NOTICE] Verified file /random_block_file_17, 134217728 bytes [NOTICE] Verified file /random_block_file_18, 134217728 bytes [NOTICE] Verified file /random_block_file_2, 134217728 bytes [NOTICE] Verified file /random_block_file_20, 134217728 bytes [NOTICE] Verified file /random_block_file_23, 134217728 bytes [NOTICE] Verified file /random_block_file_25, 134217728 bytes [NOTICE] Verified file /random_block_file_26, 134217728 bytes [NOTICE] Verified file /random_block_file_29, 134217728 bytes [NOTICE] Verified file /random_block_file_4, 134217728 bytes [NOTICE] Verified file /random_block_file_7, 134217728 bytes [NOTICE] Verified file /random_block_file_8, 134217728 bytes [INFO] Execution exec-0e89cdb8072a8bf41 finished with status Success.
次にVPC Flow LogsでDataSyncタスクのENIで発生した通信を確認します。
結果は以下の通りです。
5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.38 10.0.1.39 2049 54480 6 2 112 1660549402 1660549414 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 22 IPv4 10.0.1.38 10.0.1.39 us-east-1 use1-az6 - - - - ingress - 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.39 10.0.1.38 54480 2049 6 3 164 1660549402 1660549414 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 3 IPv4 10.0.1.39 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.38 10.0.1.39 2049 34420 6 6 5840 1660549402 1660549414 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 22 IPv4 10.0.1.38 10.0.1.39 us-east-1 use1-az6 - - - - ingress - 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.39 10.0.1.38 34420 2049 6 9 2398 1660549402 1660549414 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 7 IPv4 10.0.1.39 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.38 10.0.1.39 2049 34436 6 20 9431 1660549440 1660549442 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 18 IPv4 10.0.1.38 10.0.1.39 us-east-1 use1-az6 - - - - ingress - 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.39 10.0.1.38 34436 2049 6 25 7077 1660549440 1660549442 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 2 IPv4 10.0.1.39 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.38 10.0.1.39 2049 36476 6 2 112 1660549454 1660549464 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 22 IPv4 10.0.1.38 10.0.1.39 us-east-1 use1-az6 - - - - ingress - 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.39 10.0.1.38 36476 2049 6 3 164 1660549454 1660549464 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 3 IPv4 10.0.1.39 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.38 10.0.1.39 2049 36496 6 18 9327 1660549454 1660549464 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 18 IPv4 10.0.1.38 10.0.1.39 us-east-1 use1-az6 - - - - ingress - 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.39 10.0.1.38 36496 2049 6 25 7045 1660549454 1660549464 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 2 IPv4 10.0.1.39 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.38 10.0.1.39 2049 36488 6 6 5840 1660549454 1660549464 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 22 IPv4 10.0.1.38 10.0.1.39 us-east-1 use1-az6 - - - - ingress - 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.39 10.0.1.38 36488 2049 6 9 2398 1660549454 1660549464 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 7 IPv4 10.0.1.39 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.38 10.0.1.39 2049 34436 6 6 616 1660549498 1660549500 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 4 IPv4 10.0.1.38 10.0.1.39 us-east-1 use1-az6 - - - - ingress - 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.39 10.0.1.38 34436 2049 6 7 689 1660549498 1660549500 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 5 IPv4 10.0.1.39 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.38 10.0.1.39 2049 36496 6 235837 2029383319 1660549518 1660549520 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 0 IPv4 10.0.1.38 10.0.1.39 us-east-1 use1-az6 - - - - ingress - 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.39 10.0.1.38 36496 2049 6 45709 3062521 1660549518 1660549520 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 0 IPv4 10.0.1.39 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.38 10.0.1.39 2049 36496 6 5 373 1660549576 1660549578 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 0 IPv4 10.0.1.38 10.0.1.39 us-east-1 use1-az6 - - - - ingress - 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.39 10.0.1.38 36496 2049 6 6 469 1660549576 1660549578 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 0 IPv4 10.0.1.39 10.0.1.38 us-east-1 use1-az6 - - - - egress 1
5 <AWSアカウントID> eni-0a1f10f7b3e31df89 10.0.1.38 10.0.1.41 2049 43528 6 2 112 1660549422 1660549426 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 22 IPv4 10.0.1.38 10.0.1.41 us-east-1 use1-az6 - - - - ingress - 5 <AWSアカウントID> eni-0a1f10f7b3e31df89 10.0.1.41 10.0.1.38 43528 2049 6 3 164 1660549422 1660549426 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 3 IPv4 10.0.1.41 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-0a1f10f7b3e31df89 10.0.1.38 10.0.1.41 2049 35156 6 6 5840 1660549422 1660549426 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 22 IPv4 10.0.1.38 10.0.1.41 us-east-1 use1-az6 - - - - ingress - 5 <AWSアカウントID> eni-0a1f10f7b3e31df89 10.0.1.41 10.0.1.38 35156 2049 6 9 2396 1660549422 1660549426 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 7 IPv4 10.0.1.41 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-0a1f10f7b3e31df89 10.0.1.38 10.0.1.41 2049 35160 6 21 9756 1660549454 1660549464 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 18 IPv4 10.0.1.38 10.0.1.41 us-east-1 use1-az6 - - - - ingress - 5 <AWSアカウントID> eni-0a1f10f7b3e31df89 10.0.1.41 10.0.1.38 35160 2049 6 31 7724 1660549454 1660549464 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 2 IPv4 10.0.1.41 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-0a1f10f7b3e31df89 10.0.1.38 10.0.1.41 2049 42128 6 6 5840 1660549454 1660549464 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 22 IPv4 10.0.1.38 10.0.1.41 us-east-1 use1-az6 - - - - ingress - 5 <AWSアカウントID> eni-0a1f10f7b3e31df89 10.0.1.41 10.0.1.38 42128 2049 6 9 2396 1660549454 1660549464 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 7 IPv4 10.0.1.41 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-0a1f10f7b3e31df89 10.0.1.38 10.0.1.41 2049 42118 6 2 112 1660549454 1660549464 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 22 IPv4 10.0.1.38 10.0.1.41 us-east-1 use1-az6 - - - - ingress - 5 <AWSアカウントID> eni-0a1f10f7b3e31df89 10.0.1.41 10.0.1.38 42118 2049 6 3 164 1660549454 1660549464 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 3 IPv4 10.0.1.41 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-0a1f10f7b3e31df89 10.0.1.38 10.0.1.41 2049 42142 6 236558 2029435037 1660549492 1660549500 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 18 IPv4 10.0.1.38 10.0.1.41 us-east-1 use1-az6 - - - - ingress - 5 <AWSアカウントID> eni-0a1f10f7b3e31df89 10.0.1.41 10.0.1.38 42142 2049 6 39068 2734794 1660549492 1660549500 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 2 IPv4 10.0.1.41 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-0a1f10f7b3e31df89 10.0.1.38 10.0.1.41 2049 35160 6 4 239 1660549492 1660549500 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 4 IPv4 10.0.1.38 10.0.1.41 us-east-1 use1-az6 - - - - ingress - 5 <AWSアカウントID> eni-0a1f10f7b3e31df89 10.0.1.41 10.0.1.38 35160 2049 6 4 196 1660549492 1660549500 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 5 IPv4 10.0.1.41 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-0a1f10f7b3e31df89 10.0.1.38 10.0.1.41 2049 42142 6 4 321 1660549552 1660549554 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 0 IPv4 10.0.1.38 10.0.1.41 us-east-1 use1-az6 - - - - ingress - 5 <AWSアカウントID> eni-0a1f10f7b3e31df89 10.0.1.41 10.0.1.38 42142 2049 6 5 417 1660549552 1660549554 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 0 IPv4 10.0.1.41 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-0a1f10f7b3e31df89 10.0.1.38 10.0.1.41 2049 42142 6 5 373 1660549612 1660549614 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 0 IPv4 10.0.1.38 10.0.1.41 us-east-1 use1-az6 - - - - ingress - 5 <AWSアカウントID> eni-0a1f10f7b3e31df89 10.0.1.41 10.0.1.38 42142 2049 6 6 469 1660549612 1660549614 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 0 IPv4 10.0.1.41 10.0.1.38 us-east-1 use1-az6 - - - - egress 1
どちらのENIもEFSのIPアドレス10.0.1.38にtcp/2049でアクセスしています。また、パケット数、バイト数のどちらもそれなりの量で片方に大きく偏っている訳ではないため、転送時はActive/Activeで動作していそうです。
DataSyncタスクの実行 (2回目)
セキュリティグループの変更
Active/Activeなら片方のENIのセキュリティグループを変更して、EFSに通信できなくするとどうなるのでしょうか。
気になったので検証します。
インバウンドルールは何もなく、アウトバウンドルールは255.255.255.255/32のicmpタイプ252のポート86と存在しないトラフィックタイプに制限したセキュリティグループを用意しました。
$ aws ec2 describe-security-group-rules \
--filter Name="group-id",Values="sg-097f727fc4def0c15"
{
"SecurityGroupRules": [
{
"SecurityGroupRuleId": "sgr-035acb2fd92af1b24",
"GroupId": "sg-097f727fc4def0c15",
"GroupOwnerId": "<AWSアカウントID>",
"IsEgress": true,
"IpProtocol": "icmp",
"FromPort": 252,
"ToPort": 86,
"CidrIpv4": "255.255.255.255/32",
"Description": "Disallow all traffic",
"Tags": []
}
]
}
こちらのセキュリティグループをeni-01e89bc06f8bbb7c3にアタッチして、EFSにアクセスできなくします。
実行結果の確認
この状態でDataSyncタスクを実行します。
しばらく待つとTask failed to access location loc-070033dbd5d040458: x40016: Failed to connect to EFS mount target with IP: fs-0bf8451bb6073b581.efs.us-east-1.amazonaws.com. Please ensure that mount target's security group allows 2049 ingress from the DataSync security group or hosts within the mount target's subnet. The DataSync security group should also allow all egress to the EFS mount target and its security group.とエラーになってしまいました。
DataSyncのログもタスク開始時のログしかありませんでした。
[INFO] Request to start task-0ada18d8a999c08a0.
DataSyncのENIのVPC Flow Logsは以下の通りです。
5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.39 10.0.1.38 51304 2049 6 3 180 1660552064 1660552070 REJECT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 2 IPv4 10.0.1.39 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.39 10.0.1.38 43544 2049 6 3 180 1660552072 1660552098 REJECT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 2 IPv4 10.0.1.39 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.39 10.0.1.38 51402 2049 6 3 180 1660552072 1660552098 REJECT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 2 IPv4 10.0.1.39 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.39 10.0.1.38 38886 2049 6 3 180 1660552072 1660552098 REJECT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 2 IPv4 10.0.1.39 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.39 10.0.1.38 38896 2049 6 3 180 1660552072 1660552098 REJECT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 2 IPv4 10.0.1.39 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.39 10.0.1.38 46552 2049 6 3 180 1660552100 1660552130 REJECT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 2 IPv4 10.0.1.39 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.39 10.0.1.38 49380 2049 6 3 180 1660552100 1660552130 REJECT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 2 IPv4 10.0.1.39 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.39 10.0.1.38 49388 2049 6 3 180 1660552100 1660552130 REJECT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 2 IPv4 10.0.1.39 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.39 10.0.1.38 36092 2049 6 3 180 1660552100 1660552130 REJECT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 2 IPv4 10.0.1.39 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.39 10.0.1.38 50374 2049 6 1 60 1660552100 1660552130 REJECT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 2 IPv4 10.0.1.39 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.39 10.0.1.38 50374 2049 6 2 120 1660552130 1660552160 REJECT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 2 IPv4 10.0.1.39 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.39 10.0.1.38 54080 2049 6 3 180 1660552130 1660552160 REJECT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 2 IPv4 10.0.1.39 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.39 10.0.1.38 54086 2049 6 3 180 1660552130 1660552160 REJECT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 2 IPv4 10.0.1.39 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.39 10.0.1.38 54342 2049 6 3 180 1660552130 1660552160 REJECT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 2 IPv4 10.0.1.39 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.39 10.0.1.38 37768 2049 6 2 120 1660552130 1660552160 REJECT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 2 IPv4 10.0.1.39 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.39 10.0.1.38 37768 2049 6 1 60 1660552160 1660552162 REJECT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 2 IPv4 10.0.1.39 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 - - - - - - - 1660552220 1660552250 - NODATA vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - - - - - us-east-1 use1-az6 - - - - - -
5 <AWSアカウントID> eni-0a1f10f7b3e31df89 - - - - - - - 1660552033 1660552063 - NODATA vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - - - - - us-east-1 use1-az6 - - - - - - 5 <AWSアカウントID> eni-0a1f10f7b3e31df89 10.0.1.38 10.0.1.41 2049 35800 6 2 112 1660552083 1660552087 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 22 IPv4 10.0.1.38 10.0.1.41 us-east-1 use1-az6 - - - - ingress - 5 <AWSアカウントID> eni-0a1f10f7b3e31df89 10.0.1.41 10.0.1.38 35800 2049 6 3 164 1660552083 1660552087 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 3 IPv4 10.0.1.41 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-0a1f10f7b3e31df89 10.0.1.38 10.0.1.41 2049 35808 6 6 5840 1660552083 1660552087 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 22 IPv4 10.0.1.38 10.0.1.41 us-east-1 use1-az6 - - - - ingress - 5 <AWSアカウントID> eni-0a1f10f7b3e31df89 10.0.1.41 10.0.1.38 35808 2049 6 9 2396 1660552083 1660552087 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 7 IPv4 10.0.1.41 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-0a1f10f7b3e31df89 - - - - - - - 1660552093 1660552123 - NODATA vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - - - - - us-east-1 use1-az6 - - - - - - 5 <AWSアカウントID> eni-0a1f10f7b3e31df89 10.0.1.38 10.0.1.41 2049 35814 6 21 9483 1660552129 1660552131 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 18 IPv4 10.0.1.38 10.0.1.41 us-east-1 use1-az6 - - - - ingress - 5 <AWSアカウントID> eni-0a1f10f7b3e31df89 10.0.1.41 10.0.1.38 35814 2049 6 27 7179 1660552129 1660552131 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 2 IPv4 10.0.1.41 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-0a1f10f7b3e31df89 - - - - - - - 1660552153 1660552183 - NODATA vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - - - - - us-east-1 use1-az6 - - - - - - 5 <AWSアカウントID> eni-0a1f10f7b3e31df89 10.0.1.38 10.0.1.41 2049 35814 6 4 321 1660552189 1660552191 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 0 IPv4 10.0.1.38 10.0.1.41 us-east-1 use1-az6 - - - - ingress - 5 <AWSアカウントID> eni-0a1f10f7b3e31df89 10.0.1.41 10.0.1.38 35814 2049 6 5 417 1660552189 1660552191 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 0 IPv4 10.0.1.41 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-0a1f10f7b3e31df89 10.0.1.38 10.0.1.41 2049 42080 6 16 7754 1660552215 1660552219 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 22 IPv4 10.0.1.38 10.0.1.41 us-east-1 use1-az6 - - - - ingress - 5 <AWSアカウントID> eni-0a1f10f7b3e31df89 10.0.1.41 10.0.1.38 42080 2049 6 25 5378 1660552215 1660552219 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 7 IPv4 10.0.1.41 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-0a1f10f7b3e31df89 10.0.1.38 10.0.1.41 2049 35814 6 4 352 1660552215 1660552219 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 4 IPv4 10.0.1.38 10.0.1.41 us-east-1 use1-az6 - - - - ingress - 5 <AWSアカウントID> eni-0a1f10f7b3e31df89 10.0.1.41 10.0.1.38 35814 2049 6 5 405 1660552215 1660552219 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 5 IPv4 10.0.1.41 10.0.1.38 us-east-1 use1-az6 - - - - egress 1
セキュリティグループを変更したENInの通信は全てREJECTになっていますが、もう片方のENIの通信はACCEPTとなっています。片方だけでもENIのセキュリティグループで通信を制限してしまうとよろしくない様です。
DataSyncタスクの実行 (3回目)
ENIの削除
中途半端に通信出来ないENIがあるのがよくないのでは?と思ったので、ENIを1つ削除してみます。
# ENIを1つ削除
$ aws ec2 delete-network-interface --network-interface-id eni-0a1f10f7b3e31df89
# ENIが削除されたことを確認
$ aws ec2 describe-network-interfaces \
--filters Name=vpc-id,Values=vpc-0bc13773f6bcc9506 \
--query "NetworkInterfaces[].[NetworkInterfaceId, Description, PrivateIpAddresses, Attachment, Groups, AvailabilityZone, SubnetId]"
[
[
"eni-0944e70a73544a760",
"EFS mount target for fs-0bf8451bb6073b581 (fsmt-0463bd620e0884896)",
[
{
"Primary": true,
"PrivateDnsName": "ip-10-0-1-55.ec2.internal",
"PrivateIpAddress": "10.0.1.55"
}
],
{
"AttachmentId": "ela-attach-026a19a4058224b84",
"DeleteOnTermination": false,
"DeviceIndex": 1,
"InstanceOwnerId": "amazon-aws",
"Status": "attached"
},
[
{
"GroupName": "EfsStack-EFSFileSystemEfsSecurityGroup5F3FED7B-1J69W17R1MWCH",
"GroupId": "sg-065656efe6cc42677"
}
],
"us-east-1b",
"subnet-0917e56a46b60966f"
],
[
"eni-07e27bf8c7fc6cf7e",
"EFS mount target for fs-0bf8451bb6073b581 (fsmt-0acc33de4179e60dc)",
[
{
"Primary": true,
"PrivateDnsName": "ip-10-0-1-38.ec2.internal",
"PrivateIpAddress": "10.0.1.38"
}
],
{
"AttachmentId": "ela-attach-078b401de82a9a60a",
"DeleteOnTermination": false,
"DeviceIndex": 1,
"InstanceOwnerId": "amazon-aws",
"Status": "attached"
},
[
{
"GroupName": "EfsStack-EFSFileSystemEfsSecurityGroup5F3FED7B-1J69W17R1MWCH",
"GroupId": "sg-065656efe6cc42677"
}
],
"us-east-1a",
"subnet-07b506c0f2d7f9fff"
],
[
"eni-01e89bc06f8bbb7c3",
"datasync client for loc-070033dbd5d040458 (us-east-1:task-0ada18d8a999c08a0)",
[
{
"Primary": true,
"PrivateDnsName": "ip-10-0-1-39.ec2.internal",
"PrivateIpAddress": "10.0.1.39"
}
],
null,
[
{
"GroupName": "EfsStack-EC2InstanceInstanceSecurityGroup6CAFB487-K4RIQM1F0WIB",
"GroupId": "sg-0e64316320498bd21"
}
],
"us-east-1a",
"subnet-07b506c0f2d7f9fff"
],
[
"eni-0f5d19add01d21dba",
"",
[
{
"Primary": true,
"PrivateDnsName": "ip-10-0-1-5.ec2.internal",
"PrivateIpAddress": "10.0.1.5"
}
],
{
"AttachTime": "2022-08-15T05:53:16+00:00",
"AttachmentId": "eni-attach-044407d2f5056acb9",
"DeleteOnTermination": true,
"DeviceIndex": 0,
"NetworkCardIndex": 0,
"InstanceId": "i-0469068d54d7aeadb",
"InstanceOwnerId": "<AWSアカウントID>",
"Status": "attached"
},
[
{
"GroupName": "EfsStack-EC2InstanceInstanceSecurityGroup6CAFB487-K4RIQM1F0WIB",
"GroupId": "sg-0e64316320498bd21"
}
],
"us-east-1a",
"subnet-0e9dd33d8333d9601"
]
]
実行結果の確認
この状態でタスクを実行します。
DataSyncのステータスが起動中の時にENIを確認すると、新しいENIeni-0cd2cdbcdd10626d9が作成され、AWSアカウント471562754046のインスタンスにアタッチされていました。
$ aws ec2 describe-network-interfaces \
--filters Name=vpc-id,Values=vpc-0bc13773f6bcc9506 \
--query "NetworkInterfaces[].[NetworkInterfaceId, Description, PrivateIpAddresses, Attachment, Groups, AvailabilityZone, SubnetId]"
[
[
"eni-07e27bf8c7fc6cf7e",
"EFS mount target for fs-0bf8451bb6073b581 (fsmt-0acc33de4179e60dc)",
[
{
"Primary": true,
"PrivateDnsName": "ip-10-0-1-38.ec2.internal",
"PrivateIpAddress": "10.0.1.38"
}
],
{
"AttachmentId": "ela-attach-078b401de82a9a60a",
"DeleteOnTermination": false,
"DeviceIndex": 1,
"InstanceOwnerId": "amazon-aws",
"Status": "attached"
},
[
{
"GroupName": "EfsStack-EFSFileSystemEfsSecurityGroup5F3FED7B-1J69W17R1MWCH",
"GroupId": "sg-065656efe6cc42677"
}
],
"us-east-1a",
"subnet-07b506c0f2d7f9fff"
],
[
"eni-01e89bc06f8bbb7c3",
"datasync client for loc-070033dbd5d040458 (us-east-1:task-0ada18d8a999c08a0)",
[
{
"Primary": true,
"PrivateDnsName": "ip-10-0-1-39.ec2.internal",
"PrivateIpAddress": "10.0.1.39"
}
],
{
"AttachTime": "2022-08-16T00:25:37+00:00",
"AttachmentId": "eni-attach-08e258ae5f1f28f5c",
"DeleteOnTermination": false,
"DeviceIndex": 1,
"NetworkCardIndex": 0,
"InstanceOwnerId": "471562754046",
"Status": "attached"
},
[
{
"GroupName": "EfsStack-EC2InstanceInstanceSecurityGroup6CAFB487-K4RIQM1F0WIB",
"GroupId": "sg-0e64316320498bd21"
}
],
"us-east-1a",
"subnet-07b506c0f2d7f9fff"
],
[
"eni-0f5d19add01d21dba",
"",
[
{
"Primary": true,
"PrivateDnsName": "ip-10-0-1-5.ec2.internal",
"PrivateIpAddress": "10.0.1.5"
}
],
{
"AttachTime": "2022-08-15T05:53:16+00:00",
"AttachmentId": "eni-attach-044407d2f5056acb9",
"DeleteOnTermination": true,
"DeviceIndex": 0,
"NetworkCardIndex": 0,
"InstanceId": "i-0469068d54d7aeadb",
"InstanceOwnerId": "<AWSアカウントID>",
"Status": "attached"
},
[
{
"GroupName": "EfsStack-EC2InstanceInstanceSecurityGroup6CAFB487-K4RIQM1F0WIB",
"GroupId": "sg-0e64316320498bd21"
}
],
"us-east-1a",
"subnet-0e9dd33d8333d9601"
],
[
"eni-0cd2cdbcdd10626d9",
"datasync client for loc-070033dbd5d040458 (us-east-1:task-0ada18d8a999c08a0)",
[
{
"Primary": true,
"PrivateDnsName": "ip-10-0-1-46.ec2.internal",
"PrivateIpAddress": "10.0.1.46"
}
],
{
"AttachTime": "2022-08-16T00:25:38+00:00",
"AttachmentId": "eni-attach-07b08d7a4f945d0fa",
"DeleteOnTermination": false,
"DeviceIndex": 1,
"NetworkCardIndex": 0,
"InstanceOwnerId": "471562754046",
"Status": "attached"
},
[
{
"GroupName": "EfsStack-EC2InstanceInstanceSecurityGroup6CAFB487-K4RIQM1F0WIB",
"GroupId": "sg-0e64316320498bd21"
}
],
"us-east-1a",
"subnet-07b506c0f2d7f9fff"
],
[
"eni-0944e70a73544a760",
"EFS mount target for fs-0bf8451bb6073b581 (fsmt-0463bd620e0884896)",
[
{
"Primary": true,
"PrivateDnsName": "ip-10-0-1-55.ec2.internal",
"PrivateIpAddress": "10.0.1.55"
}
],
{
"AttachmentId": "ela-attach-026a19a4058224b84",
"DeleteOnTermination": false,
"DeviceIndex": 1,
"InstanceOwnerId": "amazon-aws",
"Status": "attached"
},
[
{
"GroupName": "EfsStack-EFSFileSystemEfsSecurityGroup5F3FED7B-1J69W17R1MWCH",
"GroupId": "sg-065656efe6cc42677"
}
],
"us-east-1b",
"subnet-0917e56a46b60966f"
]
]
AWS公式ドキュメントで実行ステータス起動中の説明を確認すると以下のように記載されていました。
同じエージェントを使用している実行中のタスクが他にないか、キューイングが有効になっていない場合、これはタスク実行の最初のフェーズです。この時点で、AWS DataSync はタスク実行を初期化します。通常このステータスは急速ですが、数分かかる場合もあります。
ここからタスク実行の初期化の中に「ENIが足りているか」、「足りていなければ作成する」という処理が含まれていることが分かります。
DataSyncのその他の実行ステータスは以下をご覧ください。
そのまましばらく待つと、実行ステータスが成功になりました。
VPC Flow Logsを確認すると、元気に両方のENIで通信していました。
5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.38 10.0.1.39 2049 43774 6 6 5840 1660609579 1660609581 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 22 IPv4 10.0.1.38 10.0.1.39 us-east-1 use1-az6 - - - - ingress - 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.39 10.0.1.38 43774 2049 6 9 2396 1660609579 1660609581 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 7 IPv4 10.0.1.39 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.38 10.0.1.39 2049 40314 6 2 112 1660609579 1660609581 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 22 IPv4 10.0.1.38 10.0.1.39 us-east-1 use1-az6 - - - - ingress - 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.39 10.0.1.38 40314 2049 6 3 164 1660609579 1660609581 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 3 IPv4 10.0.1.39 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.38 10.0.1.39 2049 43776 6 19 9379 1660609589 1660609590 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 18 IPv4 10.0.1.38 10.0.1.39 us-east-1 use1-az6 - - - - ingress - 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.39 10.0.1.38 43776 2049 6 31 7387 1660609589 1660609590 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 2 IPv4 10.0.1.39 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.38 10.0.1.39 2049 60166 6 6 5840 1660609635 1660609641 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 22 IPv4 10.0.1.38 10.0.1.39 us-east-1 use1-az6 - - - - ingress - 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.39 10.0.1.38 60166 2049 6 9 2396 1660609635 1660609641 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 7 IPv4 10.0.1.39 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.38 10.0.1.39 2049 60182 6 18 9327 1660609635 1660609641 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 18 IPv4 10.0.1.38 10.0.1.39 us-east-1 use1-az6 - - - - ingress - 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.39 10.0.1.38 60182 2049 6 23 6939 1660609635 1660609641 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 2 IPv4 10.0.1.39 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.38 10.0.1.39 2049 35134 6 2 112 1660609635 1660609641 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 22 IPv4 10.0.1.38 10.0.1.39 us-east-1 use1-az6 - - - - ingress - 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.39 10.0.1.38 35134 2049 6 3 164 1660609635 1660609641 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 3 IPv4 10.0.1.39 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.38 10.0.1.39 2049 43776 6 4 481 1660609648 1660609650 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 0 IPv4 10.0.1.38 10.0.1.39 us-east-1 use1-az6 - - - - ingress - 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.39 10.0.1.38 43776 2049 6 5 597 1660609648 1660609650 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 0 IPv4 10.0.1.39 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.38 10.0.1.39 2049 43776 6 3 187 1660609676 1660609698 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 4 IPv4 10.0.1.38 10.0.1.39 us-east-1 use1-az6 - - - - ingress - 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.39 10.0.1.38 43776 2049 6 3 144 1660609676 1660609698 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 5 IPv4 10.0.1.39 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.38 10.0.1.39 2049 60182 6 235397 2029363611 1660609676 1660609698 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 0 IPv4 10.0.1.38 10.0.1.39 us-east-1 use1-az6 - - - - ingress - 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.39 10.0.1.38 60182 2049 6 39045 2733184 1660609676 1660609698 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 0 IPv4 10.0.1.39 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.38 10.0.1.39 2049 60182 6 5 373 1660609756 1660609758 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 0 IPv4 10.0.1.38 10.0.1.39 us-east-1 use1-az6 - - - - ingress - 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.39 10.0.1.38 60182 2049 6 6 469 1660609756 1660609758 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 0 IPv4 10.0.1.39 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.38 10.0.1.39 2049 60182 6 5 373 1660609816 1660609817 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 0 IPv4 10.0.1.38 10.0.1.39 us-east-1 use1-az6 - - - - ingress - 5 <AWSアカウントID> eni-01e89bc06f8bbb7c3 10.0.1.39 10.0.1.38 60182 2049 6 6 469 1660609816 1660609817 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 0 IPv4 10.0.1.39 10.0.1.38 us-east-1 use1-az6 - - - - egress 1
5 <AWSアカウントID> eni-0cd2cdbcdd10626d9 - - - - - - - 1660609537 1660609567 - NODATA vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - - - - - us-east-1 use1-az6 - - - - - - 5 <AWSアカウントID> eni-0cd2cdbcdd10626d9 10.0.1.38 10.0.1.46 2049 33474 6 2 112 1660609577 1660609587 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 22 IPv4 10.0.1.38 10.0.1.46 us-east-1 use1-az6 - - - - ingress - 5 <AWSアカウントID> eni-0cd2cdbcdd10626d9 10.0.1.46 10.0.1.38 33474 2049 6 3 164 1660609577 1660609587 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 3 IPv4 10.0.1.46 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-0cd2cdbcdd10626d9 10.0.1.38 10.0.1.46 2049 33482 6 6 5840 1660609577 1660609587 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 22 IPv4 10.0.1.38 10.0.1.46 us-east-1 use1-az6 - - - - ingress - 5 <AWSアカウントID> eni-0cd2cdbcdd10626d9 10.0.1.46 10.0.1.38 33482 2049 6 9 2396 1660609577 1660609587 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 7 IPv4 10.0.1.46 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-0cd2cdbcdd10626d9 10.0.1.38 10.0.1.46 2049 33494 6 19 9379 1660609577 1660609587 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 18 IPv4 10.0.1.38 10.0.1.46 us-east-1 use1-az6 - - - - ingress - 5 <AWSアカウントID> eni-0cd2cdbcdd10626d9 10.0.1.46 10.0.1.38 33494 2049 6 24 7023 1660609577 1660609587 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 2 IPv4 10.0.1.46 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-0cd2cdbcdd10626d9 - - - - - - - 1660609597 1660609627 - NODATA vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - - - - - us-east-1 use1-az6 - - - - - - 5 <AWSアカウントID> eni-0cd2cdbcdd10626d9 10.0.1.38 10.0.1.46 2049 50170 6 2 112 1660609641 1660609649 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 22 IPv4 10.0.1.38 10.0.1.46 us-east-1 use1-az6 - - - - ingress - 5 <AWSアカウントID> eni-0cd2cdbcdd10626d9 10.0.1.46 10.0.1.38 50170 2049 6 3 164 1660609641 1660609649 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 3 IPv4 10.0.1.46 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-0cd2cdbcdd10626d9 10.0.1.38 10.0.1.46 2049 33494 6 5 646 1660609641 1660609649 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 0 IPv4 10.0.1.38 10.0.1.46 us-east-1 use1-az6 - - - - ingress - 5 <AWSアカウントID> eni-0cd2cdbcdd10626d9 10.0.1.46 10.0.1.38 33494 2049 6 7 858 1660609641 1660609649 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 0 IPv4 10.0.1.46 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-0cd2cdbcdd10626d9 10.0.1.38 10.0.1.46 2049 50180 6 6 5840 1660609641 1660609649 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 22 IPv4 10.0.1.38 10.0.1.46 us-east-1 use1-az6 - - - - ingress - 5 <AWSアカウントID> eni-0cd2cdbcdd10626d9 10.0.1.46 10.0.1.38 50180 2049 6 9 2396 1660609641 1660609649 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 7 IPv4 10.0.1.46 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-0cd2cdbcdd10626d9 10.0.1.38 10.0.1.46 2049 50194 6 235500 2029780942 1660609673 1660609685 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 18 IPv4 10.0.1.38 10.0.1.46 us-east-1 use1-az6 - - - - ingress - 5 <AWSアカウントID> eni-0cd2cdbcdd10626d9 10.0.1.46 10.0.1.38 50194 2049 6 45792 3068105 1660609673 1660609685 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 2 IPv4 10.0.1.46 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-0cd2cdbcdd10626d9 10.0.1.38 10.0.1.46 2049 33494 6 4 239 1660609673 1660609685 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 4 IPv4 10.0.1.38 10.0.1.46 us-east-1 use1-az6 - - - - ingress - 5 <AWSアカウントID> eni-0cd2cdbcdd10626d9 10.0.1.46 10.0.1.38 33494 2049 6 4 196 1660609673 1660609685 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 5 IPv4 10.0.1.46 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-0cd2cdbcdd10626d9 10.0.1.38 10.0.1.46 2049 50194 6 5 373 1660609733 1660609735 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 0 IPv4 10.0.1.38 10.0.1.46 us-east-1 use1-az6 - - - - ingress - 5 <AWSアカウントID> eni-0cd2cdbcdd10626d9 10.0.1.46 10.0.1.38 50194 2049 6 6 469 1660609733 1660609735 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 0 IPv4 10.0.1.46 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-0cd2cdbcdd10626d9 10.0.1.38 10.0.1.46 2049 50194 6 6 444 1660609775 1660609777 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 4 IPv4 10.0.1.38 10.0.1.46 us-east-1 use1-az6 - - - - ingress - 5 <AWSアカウントID> eni-0cd2cdbcdd10626d9 10.0.1.46 10.0.1.38 50194 2049 6 5 417 1660609775 1660609777 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 0 IPv4 10.0.1.46 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-0cd2cdbcdd10626d9 - - - - - - - 1660609777 1660609807 - NODATA vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - - - - - us-east-1 use1-az6 - - - - - - 5 <AWSアカウントID> eni-0cd2cdbcdd10626d9 10.0.1.38 10.0.1.46 2049 47146 6 14 7211 1660609833 1660609835 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 22 IPv4 10.0.1.38 10.0.1.46 us-east-1 use1-az6 - - - - ingress - 5 <AWSアカウントID> eni-0cd2cdbcdd10626d9 10.0.1.46 10.0.1.38 47146 2049 6 19 4344 1660609833 1660609835 ACCEPT OK vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - 2 IPv4 10.0.1.46 10.0.1.38 us-east-1 use1-az6 - - - - egress 1 5 <AWSアカウントID> eni-0cd2cdbcdd10626d9 - - - - - - - 1660609837 1660609867 - NODATA vpc-0bc13773f6bcc9506 subnet-07b506c0f2d7f9fff - - - - - us-east-1 use1-az6 - - - - - -
DataSyncタスクの実行 (4回目)
DataSyncタスク実行時にENIが作成されるのであれば、転送中にENIを削除するとどんな挙動をするのか気になってきました。
ということでやってみます。
DataSyncのタスクを実行して、ENIがアタッチされたことを確認します。アタッチIDとENIのIDを控えておきます。
$ aws ec2 describe-network-interfaces \
--filters Name=vpc-id,Values=vpc-0bc13773f6bcc9506 Name=description,Values="datasync client for loc-070033dbd5d040458 (us-east-1:task-0ada18d8a999c08a0)" \
--query "NetworkInterfaces[].[NetworkInterfaceId, Description, PrivateIpAddresses, Attachment, Groups, AvailabilityZone, SubnetId]"
[
[
"eni-0cd2cdbcdd10626d9",
"datasync client for loc-070033dbd5d040458 (us-east-1:task-0ada18d8a999c08a0)",
[
{
"Primary": true,
"PrivateDnsName": "ip-10-0-1-46.ec2.internal",
"PrivateIpAddress": "10.0.1.46"
}
],
{
"AttachTime": "2022-08-16T01:44:23+00:00",
"AttachmentId": "eni-attach-09f02d63b58bce24c",
"DeleteOnTermination": false,
"DeviceIndex": 1,
"NetworkCardIndex": 0,
"InstanceOwnerId": "471562754046",
"Status": "attached"
},
[
{
"GroupName": "EfsStack-EC2InstanceInstanceSecurityGroup6CAFB487-K4RIQM1F0WIB",
"GroupId": "sg-0e64316320498bd21"
}
],
"us-east-1a",
"subnet-07b506c0f2d7f9fff"
],
[
"eni-01e89bc06f8bbb7c3",
"datasync client for loc-070033dbd5d040458 (us-east-1:task-0ada18d8a999c08a0)",
[
{
"Primary": true,
"PrivateDnsName": "ip-10-0-1-39.ec2.internal",
"PrivateIpAddress": "10.0.1.39"
}
],
{
"AttachTime": "2022-08-16T01:44:22+00:00",
"AttachmentId": "eni-attach-0d6355848f173ea40",
"DeleteOnTermination": false,
"DeviceIndex": 1,
"NetworkCardIndex": 0,
"InstanceOwnerId": "471562754046",
"Status": "attached"
},
[
{
"GroupName": "EfsStack-EC2InstanceInstanceSecurityGroup6CAFB487-K4RIQM1F0WIB",
"GroupId": "sg-0e64316320498bd21"
}
],
"us-east-1a",
"subnet-07b506c0f2d7f9fff"
]
]
DataSyncの実行ステータスが転送中になったらENIのデタッチとENIの削除を行います。
# ENIのデタッチ
$ aws ec2 detach-network-interface \
--attachment-id eni-attach-09f02d63b58bce24c
$ aws ec2 detach-network-interface \
--attachment-id eni-attach-0d6355848f173ea40
# ENIの削除
$ aws ec2 delete-network-interface \
--network-interface-id eni-0cd2cdbcdd10626d9
$ aws ec2 delete-network-interface \
--network-interface-id eni-01e89bc06f8bbb7c3
# ENIが削除されたことを確認
$ aws ec2 describe-network-interfaces \
--filters Name=vpc-id,Values=vpc-0bc13773f6bcc9506 Name=description,Values="datasync client for loc-070033dbd5d040458 (us-east-1:task-0ada18d8a999c08a0)" \
--query "NetworkInterfaces[].[NetworkInterfaceId, Description, PrivateIpAddresses, Attachment, Groups, AvailabilityZone, SubnetId]"
[]
すると、実行ステータスが転送中のまま変わらなくなりました。
通常であれば20〜30秒程度で完了するのですが、10分以上待っても完了しませんでした。
S3バケット上にはDataSyncのメタデータしかありませんでした。
$ aws s3 ls s3://efsstack-bucket83908e77-ixglvpzli7rh
PRE .aws-datasync/
DataSyncのログも実行開始時のログしか出力されていませんでした。
[INFO] Request to start task-0ada18d8a999c08a0. [INFO] Execution exec-036c4830a47c58ede started. [INFO] Started logging in destination hostId: host-0d1615572f8726890 for Execution exec-036c4830a47c58ede [INFO] Started logging in destination hostId: host-0e6502b3dc4bfae6a for Execution exec-036c4830a47c58ede
また、待っている間にENIが再作成されることはありませんでした。
DataSyncタスクの実行 (5回目)
2つあるENIをどちらも削除してしまったのが良くなかったのかもしれません。
ということで転送中にENIを1つだけ削除した時の挙動を確認します。
DataSyncの実行ステータスが転送中になったらENIのデタッチとENIの削除を行います。
# ENIが作成されたことを確認
$ aws ec2 describe-network-interfaces \
--filters Name=vpc-id,Values=vpc-0bc13773f6bcc9506 Name=description,Values="datasync client for loc-070033dbd5d040458 (us-east-1:task-0ada18d8a999c08a0)" \
--query "NetworkInterfaces[].[NetworkInterfaceId, Description, PrivateIpAddresses, Attachment, Groups, AvailabilityZone, SubnetId]"
[
[
"eni-01e7fe9c1f8096e76",
"datasync client for loc-070033dbd5d040458 (us-east-1:task-0ada18d8a999c08a0)",
[
{
"Primary": true,
"PrivateDnsName": "ip-10-0-1-39.ec2.internal",
"PrivateIpAddress": "10.0.1.39"
}
],
{
"AttachTime": "2022-08-16T04:57:15+00:00",
"AttachmentId": "eni-attach-061f9d0ebf45f9e55",
"DeleteOnTermination": false,
"DeviceIndex": 1,
"NetworkCardIndex": 0,
"InstanceOwnerId": "471562754046",
"Status": "attached"
},
[
{
"GroupName": "EfsStack-EC2InstanceInstanceSecurityGroup6CAFB487-K4RIQM1F0WIB",
"GroupId": "sg-0e64316320498bd21"
}
],
"us-east-1a",
"subnet-07b506c0f2d7f9fff"
],
[
"eni-01deb5f67e8b7a744",
"datasync client for loc-070033dbd5d040458 (us-east-1:task-0ada18d8a999c08a0)",
[
{
"Primary": true,
"PrivateDnsName": "ip-10-0-1-45.ec2.internal",
"PrivateIpAddress": "10.0.1.45"
}
],
{
"AttachTime": "2022-08-16T04:57:14+00:00",
"AttachmentId": "eni-attach-00d0e2e20483e1cd5",
"DeleteOnTermination": false,
"DeviceIndex": 1,
"NetworkCardIndex": 0,
"InstanceOwnerId": "471562754046",
"Status": "attached"
},
[
{
"GroupName": "EfsStack-EC2InstanceInstanceSecurityGroup6CAFB487-K4RIQM1F0WIB",
"GroupId": "sg-0e64316320498bd21"
}
],
"us-east-1a",
"subnet-07b506c0f2d7f9fff"
]
]
# ENIのデタッチ
$ aws ec2 detach-network-interface \
--attachment-id eni-attach-061f9d0ebf45f9e55
# ENIの削除
$ aws ec2 delete-network-interface \
--network-interface-id eni-01e7fe9c1f8096e76
An error occurred (InvalidParameterValue) when calling the DeleteNetworkInterface operation: Network interface 'eni-01e7fe9c1f8096e76' is currently in use.
# ENIの確認
$ aws ec2 describe-network-interfaces \
--filters Name=vpc-id,Values=vpc-0bc13773f6bcc9506 Name=description,Values="datasync client for loc-070033dbd5d040458 (us-east-1:task-0ada18d8a999c08a0)" \
--query "NetworkInterfaces[].[NetworkInterfaceId, Description, PrivateIpAddresses, Attachment, Groups, AvailabilityZone, SubnetId]"
[
[
"eni-01e7fe9c1f8096e76",
"datasync client for loc-070033dbd5d040458 (us-east-1:task-0ada18d8a999c08a0)",
[
{
"Primary": true,
"PrivateDnsName": "ip-10-0-1-39.ec2.internal",
"PrivateIpAddress": "10.0.1.39"
}
],
null,
[
{
"GroupName": "EfsStack-EC2InstanceInstanceSecurityGroup6CAFB487-K4RIQM1F0WIB",
"GroupId": "sg-0e64316320498bd21"
}
],
"us-east-1a",
"subnet-07b506c0f2d7f9fff"
],
[
"eni-01deb5f67e8b7a744",
"datasync client for loc-070033dbd5d040458 (us-east-1:task-0ada18d8a999c08a0)",
[
{
"Primary": true,
"PrivateDnsName": "ip-10-0-1-45.ec2.internal",
"PrivateIpAddress": "10.0.1.45"
}
],
{
"AttachTime": "2022-08-16T04:57:14+00:00",
"AttachmentId": "eni-attach-00d0e2e20483e1cd5",
"DeleteOnTermination": false,
"DeviceIndex": 1,
"NetworkCardIndex": 0,
"InstanceOwnerId": "471562754046",
"Status": "attached"
},
[
{
"GroupName": "EfsStack-EC2InstanceInstanceSecurityGroup6CAFB487-K4RIQM1F0WIB",
"GroupId": "sg-0e64316320498bd21"
}
],
"us-east-1a",
"subnet-07b506c0f2d7f9fff"
]
]
ENIの削除は失敗しましたが、ENIのデタッチはできました。
この状態でDataSyncの実行ステータスを確認すると、転送中のまま動きが止まってしまいました。
S3バケットを確認すると、random_block_file_10以外のファイルは転送されていました。
$ aws s3 ls s3://efsstack-bucket83908e77-ixglvpzli7rh
PRE .aws-datasync/
2022-08-16 05:01:30 134217728 random_block_file_1
2022-08-16 05:01:30 134217728 random_block_file_11
2022-08-16 05:01:31 134217728 random_block_file_12
2022-08-16 05:01:31 134217728 random_block_file_14
2022-08-16 05:01:31 134217728 random_block_file_15
2022-08-16 05:01:31 134217728 random_block_file_16
2022-08-16 05:01:31 134217728 random_block_file_17
2022-08-16 05:01:30 134217728 random_block_file_18
2022-08-16 05:01:31 134217728 random_block_file_19
2022-08-16 05:01:30 134217728 random_block_file_2
2022-08-16 05:01:30 134217728 random_block_file_20
2022-08-16 05:01:31 134217728 random_block_file_21
2022-08-16 05:01:30 134217728 random_block_file_22
2022-08-16 05:01:31 134217728 random_block_file_23
2022-08-16 05:01:31 134217728 random_block_file_24
2022-08-16 05:01:31 134217728 random_block_file_25
2022-08-16 05:01:31 134217728 random_block_file_26
2022-08-16 05:01:31 134217728 random_block_file_27
2022-08-16 05:01:30 134217728 random_block_file_28
2022-08-16 05:01:31 134217728 random_block_file_29
2022-08-16 05:01:30 134217728 random_block_file_3
2022-08-16 05:01:30 134217728 random_block_file_30
2022-08-16 05:01:31 134217728 random_block_file_4
2022-08-16 05:01:31 134217728 random_block_file_5
2022-08-16 05:01:31 134217728 random_block_file_6
2022-08-16 05:01:30 134217728 random_block_file_7
2022-08-16 05:01:31 134217728 random_block_file_8
2022-08-16 05:01:30 134217728 random_block_file_9
DataSyncのログはENIを2つ削除したときと同じで、実行開始時のログしか出力されていませんでした。
[INFO] Request to start task-0ada18d8a999c08a0. [INFO] Started logging in destination hostId: host-01007290aaf0f1441 for Execution exec-0c55dabcb3815443b [INFO] Execution exec-0c55dabcb3815443b started. [INFO] Started logging in destination hostId: host-0b16cd9f2fe9ab56e for Execution exec-0c55dabcb3815443b
どのファイルまで転送したのかが読み取れないのは意外ですね。
DataSyncタスクのENIが複数あるのは可用性向上のためではなさそう
AWS DataSyncタスク作成時にENIが複数作成される意味に思いを馳せてみました。
転送中に1つでもENIがデタッチされると転送処理が止まるところから、どうやらDataSyncタスクのENIが複数あるのは可用性向上のためではなさそうということが分かりました。
ENIはActive/Activeで動作していたので、ENIを複数使ってスループットを向上させる意味合いが強いように思いました。
転送処理が止まってしまうので、間違ってDataSyncで転送中のENIをデタッチ・削除してしまわないように気をつけましょう。
この記事が誰かの助けになれば幸いです。
以上、AWS事業本部 コンサルティング部の のんピ(@non____97)でした!
